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Description 

BACKGROUND OF THE INVENTION 

5 [0001] The invention relates to electronic image methods and devices, and, more particularly, to digital communica- 
tion and storage systems with compressed images. 

[0002] Video communication (television, teleconferencing, Internet, digital camera, and so forth) typically transmits 
a stream of video frames (pictures, images) along with audio over a transmission channel for real time viewing and 
listening or storage. However, transmission channels frequently add corrupting noise and have limited bandwidth. 

io Consequently, digital video transmission with compression enjoys widespread use. In particular, various standards for 
compression of digital video have emerged and include the various JPEG and MPEG standards. 
[0003] Tekaip, Digital Video Processing (Prentice Hall 1995), Clarke, Digital Compression of Still Images and Video 
(Academic Press 1995), and Schafer et al, Digital Video Coding Standards and Their Role in Video Communications, 
83 Proc. IEEE 907 (1995), include summaries of various compression methods, including descriptions of the JPEG. 

is H.261 , MPEG-1 , and MPEG-2 standards 

[0004] For encoding a single frame as in JPEG or an I frame in MPEG, Shapiro, Embedded Image Coding Using 
Zerotrees of Wavelet Coefficients, 41 IEEE Tr.Sig.Proc 3445 (1993) provides a wavelet hierarchical subband decom- 
position which groups wavelet coefficients at different scales and predicts zero coefficients across scales. This provides 
a quantization and fully embedded bitstream in the sense that the bitstream of a lower bitrate is embedded in the 

20 bitstream of higher bitrates. 

[0005] Villasenor et al, Wavelet Filter Evaluation for Image Compression, 4 IEEE Tr.lmage Proc. 1053 (1995) dis- 
cusses the wavelet subband decomposition with varidus^mother wavelets. 

[0006] The loss of synchronization happens frequently when transmitting compressed bitstreams coded with variable 
length entropy codes. The ability of the decoder to reestablish synchronization is critical to the performance of the 

25 coding algorithm under noisy channels. Resynchronization markers have been used extensively for synchronization 
between encoder and decoder. For intance, MPEG4 error resilient profile uses resynchronization markers extensively 
In addition to establishing synchronization for the arithmetic decoder, resynchronization markers divide the bitstream 
into partitions that can be handled differently by the system transport layer For instance, unequal error protection can 
be used to put different protection on the different data partitions. 

30 [0007] Hardware and software implementations of the JPEG, H.261 , MPEG-1 , and MPEG-2 compression and de- 
coding exist. Further, programmable microprocessors or digital signal processors, such as the Ultrasparc or 
TMS320C6x, running appropriate software can handle most compression and decoding in real time, and less powerful 
processors may handle lower bitrate compression and decompression. 



35 SUMMARY OF THE INVENTION 



[0008] The present invention provides data partitioning and sequence resynchronization markers for the JPEG2000 
baseline coding and decoding methods. 

[0009] This has the advantages including improvement in the performance of the JPEG2000 baseline under channel 
error conditions and introduces no degradation to the performance of the baseline under error-free conditions. 



BRIEF DESCRIPTION OF THE DRAWINGS 



[0010] The drawings are schematic for clarity. 

[0011] Figures 1a-c is a block diagram of a preferred embodiment system and components. 
[0012] Figures 2a-c illustrates subband hierarchical decomposition. 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 



[0013] Figure 1a is a high level functional block diagram of a first preferred embodiment image system, and Figures 
1b-1c show details of the encoder and decoder. Error correction coding is omitted for clarity, and the decoding outside 
of the error concealment is generally the inverse of the encoding. 

[0014] The encoding will be explained with the help of an example for simplicity; thus presume an image (frame) of 
1152 rows of 1408 pixels with 8-bit pixels (-128 to + 127) and presume four scale levels in a wavelet hierarchical 
decomposition. The value of the pixel at (j,k) may be denoted x(j,k) for 0 < j < 1151 and 0 < k < 1407. 
[001 5] To begin the hierarchical decomposition, first filter the 11 52 by 1 408 frame with each of the four filters h 0 (j)h 0 
(k). MDh^k), G)h 0 (k), and h 1 G)h 1 (k), to give 1152 by 1408 filtered frames (boundary pixel values are used to extend 
the frame for computations which otherwise would extend beyond the frame). A computationally simple ho(k) function 
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equals 1/V2 at k=0,1, and is zero for all other k; h^k) equals 1/V2at k=0, -1/V2atatk= 1, 1/8V2at k=2.3, -1/SV2at k=- 
1,-2, and zero for all other k. The Villasenor article cited in the Background lists other fitter functions. The filtering is 
mathematically convolution with the functions, so h 0 is a lowpass filter in one dimension (averages over two adjacent 
pixels) and h 1 is a highpass filter in one dimension (essentially a difference of adjacent pixels). Thus the four filters are 

s two-dimensional lowpass-lowpass, lowpass -highpass, highpass-lowpass, and highpass-highpass, respectively. 

[0016] Next subsample each filtered frame by a factor of four by retaining only pixels at (j,k) with j and k both even 
integers. This subsampling will yield four 576 by 704 subarrays of wavelet coefficients, denoted LL1, LH1, HL1, and 
HH1 , respectively, with coefficient locations Q,k) relabelled for 0 < j < 575 and 0 < k £ 703. This forms tne first level of 
tne decomposition, and the four subarrays can be placed together to form a single 144 by 176 array wnich makes 

10 visualization of the decomposition simple as illustrated in Figure 2a. Thus LL1 is a lower resolution version of the 
original frame and could be used as a compressed version of the original frame. The values of the pixels in these 
filtered and subsampled images are the first level wavelet coefficients. 

[0017] The LL1, LH1, HL1 , and HH1 subarrays can be used to reconstruct the original frame by first interpolating 
each subarray by a factor of four (to restore the 1152 by 1408 size), then filtering the four 1152 by 1408 arrays with 
is filters g 0 (j)9o(k). 9oG)9i (k). 9i 0)9oCO> 9i ti)9i (k), respectively, and lastly pixelwise adding these four filtered images 
together. The functions g 0 and g n are lowpass and highpass filters, respectively, and relate to ho and h-, by go(n) = (- 
1) n h l (n) and g-,(n)= (-1) n h 0 (n). The h 0 , h,, g 0 , and g, functions are symmetric about 1/2, rather than about 0 as would 
be the case for an odd tap filter, so after reconstruction the pixel index is shifted by 1 to adjust for the two 1/2 pixel 
shifts during two filterings. 

20 [0018] The second level in the decomposition simply repeats the four filterings with the ho and h n functions plus 
subsampling by a factor of four but using the LL1 subarray as the input. Thus the four filtered subarrays are each 288 
by 352 and denoted LL2, LH2, HL2, and HH2. As before, the LL2, LH2, HL2, and HH2 can be arranged to visualize 
the decomposition of LL1 and also could be used for reconstruction of LL1 with the and g., based filters. The LH1 , 
HL1, and HH1 subarrays of first level coefficients remain unfittered. 

25 [001 9] Repeat this decomposition on LL2 by filtering with the four fitters based on ho and hi followed by subsampling 
to obtain LL3, LH3, HL3, and HH3 which are 18 by 22 coefficient subarrays. Again, LL3, LH3, HL3, and HH3 can be 
arranged to visualize the decomposition of LL2. 

[0020] Complete the hierarchical four level decomposition of the original frame by a last filtering with the four filters 
based on h 0 and h n followed by subsampling of LL3 to obtain LL4, LH4, HL4, and HH4 which are each a 144 rows of 
30 1 76 coefficients subarray. Figure 2c illustrates all of the resulting subarrays arranged to form an overall 1152 by 1408 
coefficient array. Figure 2c also indicates the tree relation of coefficients in various levels of the decomposition; indeed, 
a coefficient w(j,k) in LH4 is the result of filtering and subsampling of coefficients xQ.k) in LL3: 

35 w(j,k) = ho (0)hi(0)x(2j,2k) + tio < 0 ) hi ( 1 ) x (2 j , 2k-l ) 

+ ho(0)hi(-l)x(2j,2k+l) + 

ho<0)hi(2)x(2j,2k-2) 



+ h0(0)hi(-2)x<2j, 2k+2) + 

h 0 <0)hi(3)x(2j,2k-3) 

+ ho(l)hi(0)x(2j- 1,2k) + 

hoU)hi<l)x<2j- 1,2k- 1) 

+ ho (l)hi(-l)x(2j-l,2k+l) + ho (1) hi <2 )x <2j-l , 2k-2) 

+ ho (l)hi(-2)x(2j-l,2k+2) + ho(l)hi (3 )x<2 j-1, 2k-3 ) 



45 



SO 



[0021] Because the filtering plus subsampling is basically computing w(j,k) from the 2 by 2 area in LL3 (the values 
of hj(k) are small except for k=0,1), there are four coefficients (x(2j-1 ; 2k-1), x(2j-1,2k), x(2j,2k-1) and x(2j,2k)) in LL3 
which determine w(j,k) in LH4. Now each of these four coefficients in LL3 is related to a corresponding one of the four 
55 coefficients in LH3 at the same positions ((2j- 1, 2k-1), (2j-1.2k), (2j,2k-1) and (2j,2k)) because they were computed 
from essentially the same 4 by 4 location in LL2. Thus the coefficient w(j,k) in LH4 is called the parent of the four related 
coefficients z(2j-1 ,2k-1), z(2j-1,2k), z(2j,2k-1) and z(2j,2k) in LH3 and each of these four coefficients in LH3 is a child 
of the parent coefficient in LH4. This terminology extends to LH2 and LH1. The generic term descendant includes 
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children, children of children, and so forth. See Figure 2c showing descendants. Note that further decomposition levels 
could be used. 

[0022] Using the hierarchical decomposition of the original frame into coefficient subarrays LL4, LH4, HH1 , classify 
the wavelet coefficients blocks of block size 8x8 in the next to last subbands according to the variance of the energy, 
of the coefficients. The classification results are then propagated to the other subbands by descendency. Note that 
other classifications could be used. 

[0023] The coefficients that are in the same subband and belong to the same class are scanned block by block into 
a 1 -dimensional sequence. 

[0024] Each sequence is then quantized using Trellis Coded Quantization (TCQ). The quantized coefficients are 
then coded bitplane by bitlane with arithmetic coding. 

[0025] The preferred embodiment data partitioning is based on the data unit of a sequence. At the beginning of each 
sequence, initialize the models used in the binary arithmetic coder with a uniform distribution. This initialization cuts 
off any cross-sequence dependencies and is essential for robust decoding. Experimental results also show that the 
re-initialization does not degrade the performance for the error-free case. 

[0026] A resynchronization marker is put at the beginning of the bitstream for each sequence. The sequence ^syn- 
chronization marker has three bytes and can be described as follows: 



20 



Sequence 


Sequence 


Resynch Marker 


Number 



[0027] The first two bytes are the sequence resynchronization marker that is not emulated by the arithmetic coder. 
^ The sequence number enables the decoder to do error detection 'line! concealment intelligently 

[0028] An alternative resynchronization marker can be described as follows: 



FF 



DO 



Sequence Number 



[0029] The first two bytes of this alternative marker are the JPEG resynchronization marker that is not emulated by 
the arithmetic coder. 

30 [0030] At the decoder side, the decoder starts from the first sequence. After it finds a sequence resynchronization 
marker, it initializes the statistical models with uniform distribution and starts the decoding of another sequence. 
[0031] The error detection is done by two factors: (a) the sequence number and (b) the size of the bitstream between 
two resynchronization markers: 

35 (a) Error detection by sequence number: there are two sequence numbers present to the decoder after a successful 

search for a resynchronization marker. One sequence number is the internal counter for decoded sequences, and 
another one is the number contained in the resynchronization marker. When the two are in agreement, no error 
signal is flashed. Otherwise, we know an error occurred. 

40 [0032] When an error is detected by the sequence number, estimate the true sequence number for this sequence 
to be decoded by looking for the next resynchronization marker. A true resynchronization marker can be estimated 
using the following methods: 

[0033] Let k be the internal counter value, seq_no1 be the sequence number indicated by the current resynchroni- 
zation marker, and seq_no2 be the sequence number from the next resynchronization marker. The sequence number 
45 for the current sequence can be estimated as follows: 



so 



55 
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if (k! = seq_.no 1) /* error detected */ 
{ 

5 if ( seq_no2<«seq_nol ) 

estimated_seq_.no = seq_no2-l; 
else 

estimated sea no = seq_nol; 

10 

if ( e s t im a t ed_s eq_no < k) 
estimated seq no = k; 

} 

[0034] Then do error concealment for sequence number k to estimated_seq_no-1 . Currently, for error concealment, 
we reset all the quantized values to zero. 

20 (b) Error detection by size of the bitstream: there are two sizes for each sequence. The sizes from the arithmetic 

decoder (after decoding the bitstream for the sequence), and the size estimated by checking the distance between 
two ^synchronization makers. When these two sizes are in agreement, no error signal is flashed. Otherwise, we 
have an error. When an error is detected, we do error concealment for the sequence. 

25 Experimental results 

[0035] The following Tables 1-4 shows PSNR results using the above described methods under various channel 
errors. There are two types of channel error used: random error and burst error. The channel errors are simulated 
using the software simulation package distributed to the JPEG community by NTT-Docomo. The results are from the 
30 average of 1 00 runs of simulation. Section (1 ) shows the comparisons of results between the JPEG2000 baseline and 
the preferred embodiment method under various error rates and types, and section (2) compares the performance 
under error free conditions in a bit more detail. According to the common test conditions for error resilience, the pa- 
rameters used for baseline coding are: -mallat, -bp_r, -a, -w7x9, -e 0.002 0. 



35 



45 



55 



(1) PSNR results: 
[0036] 



Table 1 . 



PSNR Results for Bike 


Bike 




0.0125 b/p 




0.5 b/p 




Burst Error 


BER 


Baseline 


Core 


Baseline 


Core 


BLEN 




Gain 




Gain 




0.001 


1 


5.92 
16.15 


10.23 


6.55 
15.38 


8.83 


0.001 


10 


8.74 
22.35 


13.61 


6.82 
21.03 


14.21 


0.01 


1 


5.62 

11.43 

5.81 




6.31 
11.34 


5.03 


0.01 


10 


6.26 

15.81 

9.55 




7.49 
15.05 


7.56 
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Table 1. (continued) 





PSNR Results for Bike 




Bike 




0.0125 b/p 




0.5 b/p 




5 


Burst Error 


BER 


Baseline 


Core 


Baseline 


Core 




BLEN 




Gain 










ndnooin 


Error 


O. DO 








to 


0.001 




11.21 




11.15 


5.13 






5.55 














5.33 




5.51 






0.01 




10.95 




10.98 


5.47 


15 






5.62 












4.38 




4.56 






0.1 




10.98 
6.60 




10.98 


6.42 


20 


Error Free 




28.77 




32.56 








28.79 


+0.02 


32.56 


0.00 



Table 2. 
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PSNR Results for Cafe 


Cafe 




0.0125 b/p 




0.5 b/p 




Burst Error 


BER 


Baseline 


Core 


Baseline 


BLEN 




Gain 




Core 


Gain 


0.001 


1 


5.83 
14.64 


8.81 


6.41 
13.96 


7.55 


0.001 


10 


7.01 
18.93 


11.92 


6.66 
18.94 


12.26 


0.01 


1 


5.64 
11.33 


5.69 


6.14 
11.15 


5.01 


0.01 


10 


5.76 
14.42 


8.66 


6.33 
13.76 


7.43 


Random 
0.001 


Error 


5.57 
11.15 


5.58 


5.87 
11.03 


5.16 


0.01 




5.35 
10.89 


5.54 


5.69 
10.85 


5.16 


0.1 




5.44 
10.89 


5.45 


4.87 
10.88 


6.01 


Error Free 




23.13 
23.11 


-0.02 


26.68 
26.69 


+0.01 
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Table 3. 



s 



1S 



20 



25 



PSNR Results for Woman 


Woman 




U.U1 25 D/p 




0.5 b/p 




Burst Error 


BER 


Baseline 


Core 


Baseline 


Core 


BLEN 




Gain 




Gain 




0.001 


1 


8.80 
20.38 


11.58 


9.52 
19.50 


9.98 


0.001 


10 


12.68 
25.69 


13.01 


10.55 
25.60 


15.05 


0.01 


1 


7.42 
12.85 


5.43 


7.66 
12.67 


5.01 


0.01 


10 


8.87 
19.91 


11.04 


10,48 
19.07 


8.59 


Random 
0.001 


Error 


7.23 
12.55 


5.32 


7.43 
12.38 


4.95 


0.01 




6.33 
12.08 


5.75 


6.45 
12.06 


5.61 


0.1 




5.09 
12.09 


7.00 


5.17 

12.06 


6.89 


Error Free 




27.52 
27.52 


0.00 


33.78 
33.78 


0.00 



(2). Comparisons for error-free cases: 
[0037] 

Table 4. 
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40 



45 



50 



55 



Performance for error-free cases 


Image 


Baseline 

Bitrate 
PSNR 


Core-Experiment 

Bitrate 
PSNR 


Bike 


1.9999 


44.04 


1.9996 


44.04 




0.9993 


37.38 


0.9996 


37.39 




0.4994 


32.56 


0.4995 


32.56 




0.2498 


28.77 


0.2499 


28.79 




0.1242 


25.62 


0.1241 


25.62 




0.0617 


23.14 


0.0619 


23.14 


Cafe 


1.9998 


39.15 


1.9994 


39.14 




1.0000 


31.80 


0.9992 


31.79 




0.4984 


26.68 


0.4998 


26.69 




0.2499 


23.13 


0.2496 


23.11 




0.1246 


20.61 


0.1242 


20.59 




0.0612 


18.72 


0.0615 


18.72 


Woman 


1.9992 


44.74 


1.9997 


44.73 
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Table 4. (continued) 





Performance for error-free cases 


5 


Image 


Baseline 

Bitrate 
PSNR 


Core-Experiment 

Bitrate 
PSNR 






u.yyyy 


OQ A C 
J0.40 


0.9995 


38.44 






u.4yco 


*5Q 7Q 


u.4yoo 


OO ~7Q 


10 




n OAQQ 

u.^4yo 


nn on 


U.^493 


30.19 






U. l £Jo 


0"7 CO 


0.1 240 


27.52 






r\ ncoo 
U.UD^J 


£3.34 


r\ neon 
U.UO^U 


25.51 


IS 


AeriaI2 


i .yyoi 


OQ QO 

jy.oo 


i OOQ7 


39.83 






1 .UUUU 


Qji no 

J4.uy 


0.9995 


34.08 






u.4yyo 


oU.y4 


u.4yyo 


on oa 

oU.y4 






n O/IQO 


OQ CO 


n OvlO>1 

U.^434 


OO CO 


20 




U. 1 £.*r\ 


OC .AO. 


n 1 Ovi o 
U. 1 ^4o 


oc >to 






n nRi a 


OA OP 


U.UO I o 


OA OQ 
^4.£0 




Target 


1.9994 


54.1 9 


1.9982 


54.39 


25 




0.9990 


39.55 


0.9991 


39.65 






0.4987 


30.36 


0.4994 


30.50 






0.2494 


24.43 


0.2497 


24.47 


30 




0.1243 


19.71 


0.1240 


16.99 




0.0624 


16.49 


0.0619 


16.09 




Compnd2 


1.9996 


51.08 


1.9998 


51.08 






0.9993 


41.57 


0.9993 


41.58 


35 




0.4991 


34.52 


0.4991 


34.53 






0.2494 


29.66 


0.2493 


29.66 






0.1245 


26.17 


0.1243 


26.17 






0.0621 


23.43 


0.0619 


23.48 



[0038] The foregoing results clearly show that the data partitioning and ^synchronization method of the preferred 
embodiment significantly improves the performance of the baseline coder under various channel conditions. The im- 
provement in PSNR is typically more than 10db for burst errors. In the meantime, the coding efficiency under error- 
free condition remains the same. 

Syntax Changes 

[0039] The preferred embodiment data partitioning and ^synchronization method thus improve the current 
JPEG2000 baseline error resiliency. In addition, since for the error-free cases the preferred embodiment changes have 
the same coding efficiency as the current baseline for ail types of images at all bitrates, the preferred embodiment 
syntax should be incorporated into the current JPEG2000 baseline syntax. 

[0040] In addition to the above preferred embodiment data partitioning and resynchronization, which has been shown 
to significantly improve the robustness of the baseline, the following preferred embodiment syntax, which should further 
improve the robustness of the coder: 

(1). Bitplane level resynchronization (optional): put a bitplane resynchronization marker at the beginning of each 
bitplane, and reinitialize the arithmetic models to uniform distributions. The format should be as follows: 
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Bitptane 


Bitplane 


Resynch Marker 


Number 



(2) . Block level resynch ronization (optional): put a resynch ronization marker at a block boundary (chosen by the 
encoder) and reinitialize the arithmetic models to uniform distributions. The format should be as follows: 





Block Resynch 


Incremental 


10 


Marker 


Block count 



IS 



[0041] The resynch marker contains the number of blocks between the current and last resynch ronization marker. 
The block size will be 8x8 at the highest subband and decrease accordingly when resolution gets lower (the same as 
the map information). 

(3). The choice of ^synchronization markers: the following are preferred values for the three resynch ronization 
markers, which will not be emulated by the arithmetic coder: 



25 



sequence resynch marker FF F0 
Bitplane resynch marker. FF DD 
Block resynch marker FF 55 

(4). The placement of various ^synchronization markers: due to the increased number of bitplanes and blocks, 
the overhead for these resynch ronization markers are not negligible and should be adapted to the channel error 
rate. A preferred embodiment makes the sequence resynch ronization marker mandatory (due to the fact that its 
impact on coding efficiency is negligible), and make the placement of bitplane and block resynchronization markers 
optional. According to the channel error rate, the encoder can decide whether to use bitplane resynchronization 
marker or block resynchronization marker, and where to place these markers. The presence of these resynchro- 
nization marker is indicated by the header information, and the decoder can determine the locations of these 
resynchronization markers by searching for them. For instance, one strategy to place the block resynchronization 
marker is to place them at equal bit intervals. 

Modifications 

[0042] The preferred embodiments may be varied in many ways while retaining one or more of their features of 
resynchronization markers for the sequences and initializing the arithmetic coding model distribution for each sequence. 



Claims 

40 

1. A method of encoding an image, comprising the steps of: 

(a) classify wavelet coefficient blocks into classes; 

(b) scan the coefficients in each class to form a sequence corresponding to the class; and 
45 (c) provide resynchronization marker for each sequence. 

2. The method of claim 1 . wherein: 

(a) encode said sequences with an arithmetic code with uniform distribution initialization of the arithmetic 
50 coding models for each sequence. 

3. A method of decoding a bitstream, comprising the steps of: 

(a) interpret a resynchronization marker in a bitstream as indication of the start of a sequences of coded 
55 coefficients constituting a class. 

4. The method of claim 3, wherein: 
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(a) the ^synchronization marker includes a sequence number. 

5 
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